What brain imaging can tell us about embodied meaning

Brain imaging studies of language processing (using fMRI) can indicate under what circumstances the embodied aspects of language representations become activated. In particular, the processing of language is distributed across a number of cortical centers, including not only classic language areas in association cortex (which might be involved in symbolic processing) , but also sensory and motor areas. A set of fMRI studies on visual imagery in sentence comprehension reveals both the perceptual-motor and the symbolic aspects of brain function that underlie language processing. Moreover, they indicate some of the conditions under which perceptual or motor representations are most likely to be activated. Another set of studies on word comprehension indicates that the neural signature of certain concrete semantic categories (tools and dwellings) and individual category exemplars can be identified by machine learning algorithms operating on the fMRI data, and that perceptual and motor representations constitute part of the signature. Visual imagery in sentence comprehension Many types of thinking, in particular language comprehension, entail the use of mental imagery. Understanding a text on architecture or automobile design seems impossible without mental imagery. Language often refers to perceptually-based information. For example, to evaluate a sentence like The number eight when rotated 90 degrees looks like a pair of spectacles, a reader must process the content of the sentence, retrieve a mental image of the shape of the digit 8, mentally apply a rotation transformation to it, and then evaluate the resulting image. In this case, there seems little doubt that a perceptually-based representation is involved in the comprehension. This perceptually-based representation has sometimes been called the referential representation or the situation model. Our fMRI studies attempt to determine the characteristics of such representations and the conditions under which they are likely to be evoked or activated. Previous studies have indicated that mental imagery generated by verbal instructions and by visual encoding activate similar cortical regions (Mellet et al., 1996, 1998, & 2002; Mazoyer et al., 2002). Several studies examining mental imagery have observed activation of the parietal area (Just et al., 2001; Mellet et al., 1996 & 2000; Deiber et al., 1998; Ishai et al., 2000, Kosslyn et al., 1993, 1996 & 1999), particularly around the intra-parietal sulcus. Our imagery studies attempted to determine the conditions under which such activation occurs during language comprehension. There is also a possibility that the neural activity underlying the imagery in language processing is affected by the presentation modality of the language (i.e., written vs. spoken). For example, the neural activity elicited in primary visual cortex during mental imagery following verbal vs. visual encoding was different (Mellet et al., 2000); there was less primary visual activation during imagery after visual encoding compared to verbal encoding, suggesting that presentation modality may indeed affect later imagery processing. Another study by Eddy and Glass (1981) examined how the visual processes in reading might be related to the visual imagery processes that a sentence engenders, comparing visual and auditory sentence presentation modes. High-imagery sentences took longer to verify as true or false than low-imagery sentences when the sentences were presented visually, but not when they were presented auditorally. These findings again suggest that the presentation modality of In M. de Vega, A. Glenberg, & A. Graesser (Eds.), 2008 Neural embodiment of sentence and word meaning Symbols and Embodiment: Debates on meaning and cognition (pp. 75-84) Oxford, UK: Oxford University Press. 2 a sentence may affect the processing of the subsequent imagery. Our imagery studies examined mental imagery processes in the context of a language comprehension task (Just et al., 2004). One of the main goals was to examine the interaction between two somewhat separable neural systems, the mental imagery and language processing systems. In the context of the embodiment debate, these studies ask not whether embodied (perceptual or motor) activation occurs, but the circumstances under which it occurs and how it is related to other more symbolic activation. To accomplish this goal, we used fMRI to measure not only the activation levels but also the functional connectivities of the regions believed to be involved in mental imagery, to determine the relations between the two systems. A second goal was to examine the effect of input modality, comparing the effect on the imagery-related activation when the sentences were either heard or read. The study examined brain activation while participants read or listened to high-imagery sentences like The number eight when rotated 90 degrees looks like a pair of spectacles or lowimagery sentences, and judged them as true or false. They included sentences requiring various types of spatial transformation or spatial assessment such as mental rotation (like the spectacles sentence), evaluation of spatial relations (e.g., On a map, Nevada is to the right of California ), combination of shapes (e.g., The number nine can be constructed from a circle and a horizontal line, a false example), and comparison of visual aspects of common objects (e.g., In terms of diameter, a quarter is larger than a nickel, which is larger than a dime). Although these sentences generally required that a spatial transformation be mentally performed, pilot studies indicated that understanding a complex spatial description without a transformation produced similar results. The low-imagery sentences could be verified by referring to general knowledge, without the use of imagery (e.g., Although they are now a sport, marathons started with Greek messengers bringing news). The sentence imagery manipulation affected the activation in regions (particularly the left intraparietal sulcus) that activate in other mental imagery tasks, such as mental rotation. Both the auditory and visual and auditory presentation experiments indicated much more activation of the intraparietal sulcus area in the high-imagery condition as shown in Figure 1, suggesting a common neural substrate for languageFigure 1. The activation in the intraparietal sulcus area is higher for high imagery sentences (particularly in the left hemisphere), and the effect is similar regardless of whether the sentences are presented visually or